Documentation through a remote presence robot

ABSTRACT

A robotic system that is used in a tele-presence session. For example, the system can be used by medical personnel to examine, diagnose and prescribe medical treatment in the session. The system includes a robot that has a camera and is controlled by a remote station. The system further includes a storage device that stores session content data regarding the session. The data may include a video/audio taping of the session by the robot. The session content data may also include time stamps that allow a user to determine the times that events occurred during the session. The session content data may be stored on a server that accessible by multiple users. Billing information may be automatically generated using the session content data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/472,277, filed Aug. 28, 2014, pending, which is a continuation ofU.S. application Ser. No. 12/362,454, filed Jan. 29, 2009, now U.S. Pat.No. 8,849,680.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The subject matter disclosed generally relates to a robotictele-presence system.

2. Background Information

Robots have been used in a variety of applications ranging from remotecontrol of hazardous material to assisting in the performance ofsurgery. For example, U.S. Pat. No. 5,762,458 issued to Wang et al.discloses a system that allows a surgeon to perform minimally invasivemedical procedures through the use of robotically controlledinstruments. One of the robotic arms in the Wang system moves anendoscope that has a camera. The camera allows a surgeon to view asurgical area of a patient.

There has been marketed a mobile robot introduced by InTouch-Health,Inc., the assignee of this application, under the trademark RP-7. TheInTouch robot is controlled by a user at a remote station. The remotestation includes personal computer with a joystick that allows the userto remotely control the movement of the robot. Both the robot and remotestation have cameras, monitors, speakers and microphones to allow fortwo-way video/audio communication.

The InTouch RP-7 system is used by medical personnel to remotely “visit”a patient. The system is particularly useful for medical specialist. Forexample, medical personnel specializing in patient stroke care canremotely examine, diagnose and prescribe a patient management plan. Withthe proliferation of such robots it would be desirable to track andstore data related to tele-presence sessions.

BRIEF SUMMARY OF THE INVENTION

A robotic system with a robot that has a camera and a remote stationcoupled to the robot. The remote station controls the robot in a sessionthat results in session content data. The system further includes astorage device that stores the session content data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of a robotic system;

FIG. 2 is an illustration showing a user interface;

FIG. 3 is an illustration of a user interface displaying events andassociated time stamps;

FIG. 4 is an illustration of a user interface with selectable fields;

FIG. 5 is an illustration showing the display of a pull-down menu;

FIG. 6 is an illustration showing a session field displayed in responseto the selection of a field;

FIG. 7 is a schematic of an electrical system of a robot;

FIG. 8 is a graphical user interface of a user interface;

FIG. 9 is a graphical user interface at a remote station;

FIG. 10 is a graphical user interface at the remote station;

FIG. 11 is a graphical user interface when a NIHSS tab is selected;

FIG. 12 is a graphical user interface displayed when a t-PA table isselected

FIG. 13 is a graphical user interface displayed when a view imagesbutton is selected.

DETAILED DESCRIPTION

Disclosed is a robotic system that is used in a tele-presence session.For example, the system can be used by medical personnel to examine,diagnose and prescribe medical treatment in the session. The systemincludes a robot that has a camera and is controlled by a remotestation. The system further includes a storage device that storessession content data regarding the session. The data may include avideo/audio taping of the session by the robot. The session content datamay also include time stamps that allow a user to determine the timesthat events occurred during the session. The session data may be storedon a server that is accessible to multiple users. Billing informationmay be automatically generated using the session data.

Referring to the drawings more particularly by reference numbers, FIG. 1shows a robotic system 10. The robotic system 10 includes one or morerobots 12. Each robot 12 may have a base station 14. The robot 12 iscoupled to a remote control station 16. The remote control station 16may be coupled to the base station 14 through a network 18. By way ofexample, the network 18 may be either a packet switched network such asthe Internet, or a circuit switched network such has a Public SwitchedTelephone Network (PSTN) or other broadband system. The base station 14may be coupled to the network 18 by a modem 20 or other broadbandnetwork interface device. By way of example, the base station 14 may bea wireless router. Alternatively, the robot 12 may have a directconnection to the network 18 through, for example, a satellite.

The remote control station 16 may include a computer 22 that has amonitor 24, a camera 26, a microphone 28 and a speaker 30. The computer22 may also contain an input device 32 such as a joystick or a mouse.The control station 16 is typically located in a place that is remotefrom the robot 12. Although only one remote control station 16 is shown,the system 10 may include a plurality of remote stations. In general anynumber of robots 12 may be controlled by any number of remote stations16 or other robots 12. For example, one remote station 16 may be coupledto a plurality of robots 12, or one robot 12 may be coupled to aplurality of remote stations 16, or a plurality of robots 12.

Each robot 12 includes a movement platform 34 that is attached to arobot housing 36. The robot 12 may also have a camera 38, a monitor 40,a microphone(s) 42 and a speaker(s) 44. The microphone 42 and speaker 30may create a stereophonic sound. The robot 12 may also have an antenna46 that is wirelessly coupled to an antenna 48 of the base station 14.The system 10 allows a user at the remote control station 16 to move therobot 12 through operation of the input device 32. The robot camera 38is coupled to the remote monitor 24 so that a user at the remote station16 can view someone at the robot site such as a patient. Likewise, therobot monitor 40 is coupled to the remote camera 26 so that someone atthe robot site can view the user. The microphones 28 and 42, andspeakers 30 and 44, allow for audible communication between the robotsite and the user of the system.

The remote station computer 22 may operate Microsoft OS software andWINDOWS XP or other operating systems such as LINUX. The remote computer22 may also operate a video driver, a camera driver, an audio driver anda joystick driver. The video images may be transmitted and received withcompression software such as MPEG CODEC.

The system 10 can be used to engage in a session that results in data.For example, the system 10 can be used by medical personnel to remotelyexamine, diagnose and prescribe a patient management plan for a patient50 in a medical session. Either the patient, or a bed supporting thepatient, may have a radio frequency information device (“RFID”) 52. TheRFID 52 may wirelessly transmit information that is received by therobot 12 through antennae 46. The RFID information can be used tocorrelate a particular session with a specific patient. The receipt ofRFID information may initiate the storage of session data. Although amedical session is described, it is to be understood that other types ofsessions may be conducted with the system 10. For example, the system 10may be used to move the robot(s) about a factory floor wherein the userprovides remote consultation. Consultation session data may be stored bythe system 10.

The system can store and display session content data. Session contentdata is information regarding the substance of a session. For example,in a medical application, session content data would include physiciannotes, diagnosis and prescription information. In a factory-equipmentrepair application, session content data would include repairmethodology and replaced parts. Session content data would not be meretime entries associated with the logging on and termination of a robotsession.

The system 10 may include a records server 54 and/or a billing server 56that can be accessed through the network 18. The servers 54 and 56 mayinclude memory, processors, I/O interfaces and storage devices such ashard disk drives, as is known in the art. Records server 54 may have astorage device(s) 57 that stores session data. The server 54 may receiveand store session data during a session. For example, the server 54 mayreceive and store video and audio captured by the robot camera 38 andmicrophone 42, respectively. To reduce bandwidth requirements during asession the session data, such as video/audio segments, can betransmitted from the robot 12 to the server 54 after the session hasterminated. For example, when the user logs off the system. Timestampedprogress notes are also simultaneously uploaded. The server 54 maycontain other medical records of a patient such as written records oftreatment, patient history, medication information, laboratory results,physician notes, etc. Video/audio segments can be timestamped andassociated with the identification of the control station and the robot,and a unique identifier which can be cross-referenced with progressnotes and other session data. These video/audio segments can then laterbe used to substantiate and reference the various progress notes andother events in a visual fashion. The system can track all head and basemovements made during the course of the associated portion of thesession, to allow correlation of those movements with the actions taken.

The system 10 may include a user interface 58 that allows a user at theremote location to enter data into the system. For example, theinterface 58 may be a computer or a computer terminal that allows a userto enter information about the patient. The robot 12 can be moved intoview of the patient through the remote station 16 so that patientinformation can be entered into the system while a physician is viewingthe patient through the robot camera. The physician can remotely movethe robot 12 to obtain different viewing angles of the patient. The userinterface 58 may be a separate computer and/or be integral with therobot 12. The billing server 56 may automatically generate a bill fromthe information provided by the session data on a periodic basis. Thebilled elements may be based on either actions performed or outcomesachieved, or both. Alternatively, a user can manually generate billsthrough a user interface to the billing server.

The billing server 56 may receive session data during a session or upontermination of a session. Additionally, the billing server may poll arobot to retrieve data from its hard drive. The session data may beorganized so as to automatically populate certain fields of a billingstatement or report. The billing information can be automatically sentto an insurance carrier.

The server 54 can be accessible through a web page or other means foraccessing information through a network 18. FIG. 2 shows a userinterface 62 displayed at a remote station 16, or any other terminalthat can access the server 54. The interface 62 can for example, providea date and time that various physicians had sessions with differentpatients. FIG. 3 shows another user interface 64 that displays timestamps 66 that are associated with certain events 68. Records can beretrieved by various filters including physician name, patient name,time of session and services performed during the session. The eventdata can be initially stored in either the robot 12 or the remotestation 16 and then loaded into the server 54, either during or after asession. Alternatively, event data can be directly loaded into theserver without storing it locally on the robot or remote station.

The session data can be organized into a plurality of data types. FIG. 4shows a plurality of different data types. For example, the session datacan be organized into ENCOUNTER PROFILE data 70, PATIENT PROFILE data 72and CLINICAL MANAGEMENT PROFILE data 74, with each having subfields suchas EVENT and HISTORY. FIG. 5 shows a pull-down screen 78 that isdisplayed when a DEMOGRAPHICS field 76 is selected. FIG. 6 shows a field80 that displays a number of sessions that match a selected HISPANICfield 82. The session data can be searched with Boolean operators suchas AND and OR to search for multiple terms, data types, etc. The usercan display all hits for the search, or have a statistical analysisperformed based on the matching sessions.

In a factory equipment-repair application, the equipment being repairedduring the session would replace the patient name in FIG. 2; and stepsfor repair would replace the event list in FIG. 3. Repair methodologiesand affected part numbers would replace the search criteria in FIGS. 4,5 and 6. Captured video and audio would show the steps in the repairprocess, and would be timestamped and cross-referenced to the data inFIG. 3.

Referring to FIG. 1, the system 10 may also include an image server 84and a registry server 86. The image server 84 may include medicalimages. For example, the medical images may include CT scans of apatient's brain. The images can be downloaded to one of the remotestations 16 through the network 18. The registry server 86 may storehistorical data on patients. The historical data can be downloaded to aremote computer 16 through the network 18.

FIG. 7 shows an embodiment of a robot 12. Each robot 12 may include ahigh level control system 160 and a low level control system 162. Thehigh level control system 160 may include a processor 164 that isconnected to a bus 166. The bus is coupled to the camera 38 by aninput/output (I/O) port 168, and to the monitor 40 by a serial outputport 170 and a VGA driver 172. The monitor 40 may include a touchscreenfunction that allows a user to enter input by touching the monitorscreen.

The speaker 44 is coupled to the bus 166 by a digital to analogconverter 174. The microphone 42 is coupled to the bus 166 by an analogto digital converter 176. The high level controller 160 may also containrandom access memory (RAM) device 178, a non-volatile RAM device 180 anda mass storage device 182 that are all coupled to the bus 172. The RAM178, NVRAM 180 and/or mass storage device 182 may contain session datathat is transmitted to the remote station and/or server. The robotantennae 46 may be coupled to a wireless transceiver 184. By way ofexample, the transceiver 184 may transmit and receive information inaccordance with IEEE 802.11b.

The controller 164 may operate with a LINUX OS operating system. Thecontroller 164 may also operate MS WINDOWS along with video, camera andaudio drivers for communication with the remote control station 16.Video information may be transceived using MPEG CODEC compressiontechniques. The software may allow the user to send e-mail to thepatient and vice versa, or allow the patient to access the Internet. Ingeneral the high level controller 160 operates to control communicationbetween the robot 12 and the remote control station 16.

The high level controller 160 may be linked to the low level controlsystem 162 by a serial port 186. The low level control system 162 mayinclude components and software that mechanically actuate the robot 12.For example, the low level control system 162 provides instructions toactuate the movement platform to move the robot 12. The low levelcontrol system 162 may receive movement instructions from the high levelcontroller 160. The movement instructions may be received as movementcommands from the remote control station or another robot. Although twocontrollers are shown, it is to be understood that each robot 12 mayhave one controller, or more than two controllers, controlling the highand low level functions.

The system may be the same or similar to a robotic system provided bythe assignee InTouch Technology, Inc. of Santa Barbara, Calif. under thename RP-7, which is hereby incorporated by reference. The system mayalso be the same or similar to the system disclosed in U.S. Pat. No.7,292,912, which is hereby incorporated by reference.

FIG. 8 shows a graphical user interface 250 can be provided at the userinterface 58. The graphical user interface 250 includes a plurality ofdata fields 252 that can be filled by the user. The data fields 252 canrequest patient information such as name, age, etc. The data fields mayalso include request for medical data such as heart rate, glucose leveland blood pressure (“SBP” and “DBP”). The data entered into the fields252 can be included in the session data that is transmitted and storedby the system 10. Filling the data fields may be designated an “event”that is given as associated time stamp and displayed by a userinterface.

FIG. 9 shows a display user interface (“DUI”) 260 that can be displayedat the remote station 16. The DUI 260 may include a robot view field 262that displays a video image captured by the camera of the robot. The DUI260 may also include a station view field 264 that displays a videoimage provided by the camera of the remote station 16. The DUI 260 maybe part of an application program stored and operated by the computer 22of the remote station 16. The video and any accompanying audio displayedby the robot and station view fields may be transmitted and stored bythe system 10 as session data.

The DUI 260 may contain a “progress notes” text editing field, whichenables a “document as you treat” methodology. As the physician conductstreatment, he can document both the treatment steps and outcomes in theprogress notes field. Each note may be manually timestamped by thephysician, or automatically timestamped by the software based on whenthe physician began typing each note. In the application of factoryfloor equipment repair, the progress notes would detail the variousexaminations and repair steps taken.

FIG. 10 shows a graphical user interface 270 that can be displayed bythe monitor of the remote station 16. The interface 270 includes a“PATIENT INFO” tab 272, a “NIHSS” tab 274 and a “t-PA” tab 276.Selection of the PATIENT INFO tab 272 displays various data fields 278including patient name, age, weight, heart rate, etc. This may be thesame information entered through the user interface 250. Thisinformation may be included in the session data that is transmitted andstored by the system 10. The usage of this interface may be tagged as anevent with an associated time stamp.

FIG. 11 shows an interface 280 when the “NIHSS” tab 274 is selected. Theinterface 280 has a data field 282 that provides a questionnaire to ratethe severity of a stroke victim using the NIHSS stroke scale. Thisprovides a readily available medical tool for the physician. The resultsof the questionnaire can be included in the session data and be taggedas an event that has an associated time stamp.

FIG. 12 shows an interface 290 when the “t-PA” tab 276 is selected. Theinterface 290 may include a data field 292 that provides the patient'sweight, a “TOTAL DOSE” data field 294, a “BOLUS DOSE” data field 296 andan “INFUSION DOSE” data field 298. The interface 290 may also include a“CALCULATE” button 300. When the CALCULATE button 300 is selected thedata fields 294, 296 and 298 are automatically populated with acalculated dosage. This provides a patient management plan for thephysician to review. The interfaces 270, 280 and 290 also have a “VIEWIMAGES” button 302 that when selected displays an interface 310 shown inFIG. 13. The interface 310 includes a data field 312 and an image field314. The image field 314 can provide a plurality of medical images suchas a CT scan of the patient's head.

The calculated dosage and images can be included in the session datathat is transmitted and stored by the system. The automatic populationof the data fields may be tagged as an event with an associated timestamp. Likewise, the selection of the data and/or image fields may betagged as events with time stamps.

The system is useful for allowing a physician to remotely view and treata stroke patient. The system provides patient information, NIHSS strokeseverity assessment, calculated t-PA dosage and CT head images thatallow the physician to provide real time remote patient treatment. Thesystem also allows such sessions to be audited so that medicalpersonnel, healthcare institutions, insurance carriers, etc. can auditsessions. Such audits may include viewing video/audio captured by therobot during a session.

While certain exemplary embodiments have been described and shown in theaccompanying drawings, it is to be understood that such embodiments aremerely illustrative of and not restrictive on the broad invention, andthat this invention not be limited to the specific constructions andarrangements shown and described, since various other modifications mayoccur to those ordinarily skilled in the art.

What is claimed is:
 1. A robotic system, comprising: a plurality of robots coupled to a network, each robot having a robot camera, a robot monitor, a robot microphone, and a robot speaker; a plurality of remote stations coupled to the network, each remote station having a station camera, a station monitor, a station microphone, and a station speaker, each remote station is capable of establishing a communication session with any one of the plurality of robots, during which session a medical personnel at the remote station has access to video from the robot camera and audio from the robot microphone, the medical personnel provides a service for a patient located in the vicinity of the robot, and the session results in session content data that is stored in a server coupled to the network, wherein the server includes a web interface accessible via a remote terminal, the web interface provides the stored session content data for display at the remote terminal, the session content data including an identifier of the patient, the medical personnel, the robot, and the service provided by the medical personnel to the patient.
 2. A robotic system, comprising: a robot that has a robot camera, a robot monitor, a robot microphone, and a robot speaker; a remote station that has a station camera, a station monitor, a station microphone, and a station speaker, said remote station and said robot are capable of establishing a telepresence session during which said station monitor is coupled to said robot camera and displays a robot image captured by said robot camera, said robot monitor is coupled to said station camera and displays a station image captured by said station camera, said station speaker is coupled to said robot microphone, and said robot speaker is coupled to said station microphone to enable two-way audio communication between said robot and said remote station, said remote station controls said robot during said telepresence session and said telepresence session results in session content data that is associated with an identifier of the robot and includes video captured by the robot camera and tracked movements of the robot; and, a storage device that stores the session content data during the telepresence session. 